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Coderivative Characterizations of Maximal 
Monotonicity for Set-Valued Mappings 

N. H. Chieu* G. M. Lee} B. S. Mordukhovich} T. T. A. Nghia § 


Abstract. This paper concerns generalized differential characterizations of maximal 
monotone set-valued mappings. Using advanced tools of variational analysis, we es¬ 
tablish coderivative criteria for maximal monotonicity of set-valued mappings, which 
seem to be the first infinitesimal characterizations of maximal monotonicity outside 
the single-valued case. We also present second-order necessary and sufficient condi¬ 
tions for lower-C 2 functions to be convex and strongly convex. Examples are provided 
to illustrate the obtained results and the imposed assumptions. 

Key Words. Maximal monotone mappings, convex lower-C 2 functions, varia¬ 
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1 Introduction 

The notion of maximal monotone operators appeared in the early 1960s and since that 
has been well recognized as a fundamental tool in the study of various aspects of partial 
differential equations, optimization, equilibrium theory, etc.; particularly those concerning 
existence and uniqueness of solutions, stability issues, convergence of numerical algorithms, 
and related topics; see, e.g., (2. HJ EJ jH[ .19] ]23j [32j ;36j [38] [39j and the references therein. We 
specially emphasize a crucial role of maximal monotonicity in the theory and applications 
of the sweeping process (“processus du rafle”) introduced and investigated by Jean Jacques 
Moreau |3D]; see more details and recent developments in the survey paper [ 12] , 

Since a single-valued and continuous monotone mapping is automatically maximal 
monotone (see, e.g., |2, Corollary 20.25]), the maximality issue does not arise in this case. 
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The classical criterion of monotonicity presented in [38] Proposition 12.3] tells us that a 
differentiable single-valued mapping is monotone if and only if its derivative is positive- 
semidefinite at every point. Infinitesimal characterizations of monotonicity for possibly 
nondifferentiable mappings have attracted much attention in the literature. In his land¬ 
mark paper [23], Minty established a sufficient condition for monotonicity of nondifferen¬ 
tiable monotone mappings by using directional derivatives. Jiang and Qi [22] Proposi¬ 
tion 2.3] and Luc and Schaible [201 Proposition 2.1] independently proved that a locally 
Lipschitzian mapping defined on an open convex set of lR n is monotone if and only if its 
Clarke’s generalized Jacobian is pointwise positive-semidefinite at every point. Replacing 
generalized Jacobian matrices by their approximate Jacobian matrices, Jeyakumar et al. 
[21 , Theorem 3.1] derived a sufficient condition for monotonicity of continuous single-valued 
mappings between finite-dimensional spaces. More recently [9], Chieu and Trang obtained 
necessary and sufficient conditions for monotonicity of continuous single-valued mappings 
in both finite-dimensional and infinite-dimensional settings via positive-semidefiniteness of 
the coderivative constructions by Mordukhovich [23] (see Sections 2), extending in this way 
the classical characterization of monotonicity for smooth mappings. 

However, the major role in nonlinear and variational analysis and their applications is 
played not by single-valued but intrinsically set-valued, maximal monotone operators that 
include, e.g., sub differential mappings for lower semicontinuous proper convex functions and 
normal cone mappings associated with close convex sets. In particular, such set-valued map¬ 
pings allow us to adequately describe variational inequalities and complementarity prob¬ 
lems in Robinson’s framework of generalized equations [33], the aforementioned Moreau’s 
sweeping process, etc. It is very attractive and challenging therefore to establish verifiable 
infinitesimal conditions (better, complete characterizations) of maximal monotonicity and 
related properties for set-valued mappings in finite and infinite dimensions. 

To the best of our knowledge, the first result in this direction was obtained by Poliquin 
and Rockafellar [33] Theorem 2.1] who derived a necessary condition for the maximal 
monotonicity of set-valued mappings between finite-dimensional spaces in terms of the 
positive-semidefiniteness of the limiting coderivative; this condition was extended in [8] [26] 
to Hilbert spaces and reversed in [9] for single-valued mappings. Note that the motivation 
of [ 32 ] came from the application to tilt stability in optimization which theory has been 
flowering during the recent years; see, e.g., |3l m OH El Mi [27] and the references therein. 

Another impact to the study and of monotonicity properties for set-valued mappings has 
been recently done by Mordukhovich and Ngliia [28] who established complete coderivative 
characterizations of strong local maximal monotonicity in finite and infinite dimensions with 
applications to full stability (in the Lipschitzian and Holderian frameworks) of parametric 
variational systems. The approach of [28] made used, along with advanced tools of vari¬ 
ational analysis and generalized differentiation, hypomonotonicity properties of set-valued 
mappings, which will be exploited in what follows. 

The main goal of this paper is to establish complete coderivative characterizations of 
maximal monotonicity of set-valued mappings in Hilbert spaces via pointwise positive- 
semidefiniteness conditions and appropriate properties of global and (semi)local hypomono¬ 
tonicity. The results obtained seem to be the first infinitesimal characterizations of max¬ 
imal monotonicity outside the single-valued setting even in the case of finite dimensions. 
As consequences of these characterizations, we derive second-order necessary and sufficient 
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conditions for lower-C 2 functions to be convex and strongly convex. These conditions are 
expressed in terms of second-order subdifferenntials/generalized Hessians of extended-real- 
valued functions and extend the classical result of real analysis saying that a C 2 function 
is convex if and only if its Hessian is positive-semidehnite at any point. 

The paper is organized as follows. Section 2 recalls some basic notions and facts from 
variational analysis that are employed in the sequel. Section 3 is the main part of our 
analysis, which contains several coderivative characterizations of maximal monotonicity 
for set-valued mappings in Hilbert spaces. Section 4 is devoted to the study of convexity 
and strong convexity for lower-C 2 functions via second-order subdifferentials. Finally, we 
present some concluding remarks and formulate open questions in Section 5. 

Our notation is standard in variational analysis and generalized differentiation; cf. [25, 
138] ). Throughout the paper we assume that A" is a Hilbert space being identified with its 
dual space. As usual, the symbol (•, -,) signifies the canonical pairing in X with the norm 
||x|| = \J(x, x ). We denote by IB the closed unit ball in X and by JB r {x ) := x + rlB the ball 
with radius r > 0 and center x G X. The notation A indicates for the weak convergence 
in X. Given a set-valued mapping F : X X and a point u G X, the symbol 

LimsupF(w) := < v G X 3 sequences Uk —> u, ry A v such that 

u^u f (1.1) 

Vk G F(uf.) for all k G IV := {1, 2,...} j 

stands for the sequential Painleve-Kuratowski outer/upper limit of F(u) as u —> u. 


2 Preliminaries 


Here we mainly follow the book [25] referring the reader also to [4] [38] for related and 
additional material. Given a proper (i.e., not identically equal to infinity) extended-real- 
valued function /: X —>■ 1R := 1R U {oo} and its domain point u G dom/ := {u G 
X| f(u) < oo}, the regular/Frechet subdifferential (known also as the presub differential 
and the viscosity subdifferential) of / at u is 


df(u) 


jv G X 


limi nf /M — /(«)-<»■ “-«> >() 


( 2 . 1 ) 


with df(u ) := 0 if u fz dom/. The limiting/Mordukhovich subdifferential (known also as 
the basic sub differential) of / at u G dom / is defined via (II. ip by 

df{u) Limsup9/(rt), (2.2) 


where the notation u -/■ u means that u —> u with f(u) —> f(u). It is well known that both 
regular and limiting sub differential reduce to the classical subdifferential of convex analysis 
when the function / is convex. On the other hand, the limiting sub differential of nonconvex 
functions and related normal cone/coderivative constructions for sets and mappings enjoy 
full calculus , which is not the case for (12.ip and its set/mapping counterparts. 
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Given further a set 0 C X with its indicator function 5(u]0) equal to 0 for u G 0 
and to oo otherwise, the regular and limiting normal cones to 0 at a 6 0 are defined, 
respectively, via the corresponding subdifferentials (12.Tj) and (12.21) by 

N(u] 0) := d8(u; 0) and N(u;0) := dd(u;0). (2.3) 

Now we consider a set-valued mapping F : X =? X and associate with it the domain 
dom F and the graph gph F by 

dorn F := {w G X\ F(u ) ^ 0} and gphF := {(rt, t>) G X x X\ v G F(u)}. 

The mapping F is said to be proper when domF 0, which is always assumed. Define by 
D*F(u, v)(w) := {z G X\ (z, — w) G N((u, v); gphF)} for all wE X (2.4) 

the regular coderivative of F at (u, v) G gph F and by 

D* m F(u,v)(w) Limsup D*F(u,v)(y) for all w G X (2.5) 

(u,v)—>(u,v) 
y—>w 

the mixed limiting coderivative of F at (u,v), where the convergence y —> w is strong in X 
while the outer limit in (12.51) is taken by (II.ip in the weak topology of X ; see [25] for more 
discussions. We omit the subscript “M” in (12.51) when X is finite-dimensional and also 
drop indicating v = F(u ) if F is single-valued. When F is single-valued and continuously 
differentiable around u (or strictly differentiable at this point), we get 

D*F(u)(w) = D* M F{u)(w) = {VF(h)*w} for all w G X 

via the adjoint derivative operator VF(a)*; see, e.g., [25], Theorem 1.38]. 

Next we recall two second-order subdifferential/generalized Hessian constructions in¬ 
troduced by the scheme suggested in [24] as a coderivative of a first-order subdifferential 
mapping; see [25, [26], [29] for more details and discussions. 

Definition 2.1 (second-order subdifferentials). Let f : X —¥ JR with u G dom/, and 
let v G df{u). Then we say that: 

(i) The COMBINED SECOND-ORDER subdifferential of f at u relative to v is the 
set-valued mapping d 2 f(x,v) : X X with the values 

d 2 f(u, v)(w) := (D*df)(u,v)(w ) for all w G X. (2.6) 

(ii) The mixed second-order subdifferential of f at u relative to v is the set¬ 
valued mapping df { f(u,v) : X X with the values 

d 2 M f{u,v)(w ) := ( D* M df)(u,v)(w) for all w G X. (2.7) 

It is worth mentioning that if / is C 2 around u with v = V/(h), then both d 2 f(u , v)(w) 
and dhf(u,v)(w ) reduce to the classical (symmetric) single-valued Hessian operator: 

d 2 f(u, v)(w) = d 2 M f{u,v){w ) = {V 2 /(h)*w} = {X 2 f(u)w} for all w G Ah 
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One of the main fact of generalized differentiation largely employed in our paper is the 
following mean value inequality for Lipschitz continuous functions; [25J Corollary 3.50(h)]. 
For the reader’s convenience we formulate it here. 

Mean-value inequality. Let f : X -A JR be a lower semicontinuous ( l.s.c .) function with 
a G dom /. Then for any b G X and £ > 0 we have the estimate 

|/(6) - /(a)| < ||6 — a|| sup {||v|| | v G df(c), cG [a, b] +eB}, (2.8) 

where [a, b] := {Aa + (1 — A)6| A G [0,1]}. 

3 Characterizations of Maximal Monotonicity 

The following notions of (global) monotonicity for set-valued mappings are main objects of 
our study and applications in this paper. 

Definition 3.1 (monotone set-valued operators). Given T \ X =$ X, we say that: 

(i) T is MONOTONE on X if 


(vi -v 2 ,ui -u 2 )> 0 for all (u h v i), (u 2 ,v 2 ) G gphT. (3.1) 

T is said to be maximal monotone on X if in addition we have gphT = gph S' whenever 
S is monotone with gphT C gph S'. 

(ii) T is hypomonotone on X if there exists a number r > 0 such that T + rl, where 
I: X —> X is the identity mapping. This means that 

(vi - v 2 ,Ui - u 2 ) > — r||-Ui - m 2 || 2 for all (tq, it), (u 2 , v 2 ) G gphT. (3.2) 

First we present a characterization of maximal monotonicity via the positive-semi definiteness 
condition for the regular coderivative and global hypomonotonicity. 

Theorem 3.2 (regular coderivative and global hypomonotonicity characteriza¬ 
tion of maximal monotonicity). Let T \ X X be a set-valued mapping with closed 
graph in the norm topology of X x X. The following assertions are equivalent: 

(i) T is maximal monotone on X. 

(ii) T is hypomonotone on X and for any (u, v) G gph T we have 

(z,w)>0 whenever z G D*T(u, v)(w). (3.3) 

Proof. Since the monotonicity obviously yields hypomonotonicity, implication (i)=^(ii) 
follows from [8j Lemma 5.2] and also from j26l Lemma 6.2]). 

To verify the converse implication (ii)=^(i), suppose that T is hypomonotone and that 
condition (13.31) is satisfied. Then there is some number r > 0 such that T + rl is monotone. 
Take any s > r and define F : X X by gph F := gph (T+s/) -1 . For any (ly, uf) G gph F, 
i = 1, 2 we have {u i: ly — suf) G gphT and thus deduce from (13.21) that 

(r>i - sui -v 2 + su 2 , U\ - u 2 > -r\\ui - m 2 || 2 . 
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The latter implies in turn that the inequalities 


IK-V 2 II • \\ui~u 2 \\ > {vi-v 2 ,u l -u 2 ) > (s — r)||u 1 — u 2 || 2 , 


which allow us to arrive at the estimate 

||«i - u 2 \\ < ■■ ||ui - v 2 

s — r 


(3.4) 


verifying that F is single-valued and Lipschitz continuous with modulus (s — r) 1 on its 
domain. To proceed further, fix any z £ X and define f z : X —> 1R by 


fz(v) 


(z,F(v )) if v £ dorn F, 
00 otherwise. 


(3.5) 


Since gph T is closed, it is easy to check that gph F is also closed in X x X. Next we 
show that f z is lower semicontinuous on X. Arguing by contradiction, suppose that there 
exist e > 0 and a sequence ry converging to some v £ X such that f z (vk ) < f z (v) ~ £ - If 
f z (y) = 00 , then v f dorn F and ty £ domF. It follows from (13.411 that ||F(ufc) — F(vf )|| < 
(s «r) -1 ||ufc — Uj||, and so {F(ufc)} is a Cauchy sequence converging to some u £ X. Hence 
the sequence (vk, F(vk)) £ gph F converges to (v, u ) £ gph F due to the closedness of gph F. 
This gives us F(v) = u and contradicts v f dom F. In the remaining case of f z {v) < 00 we 
get from (I3.4[) and (13. 5 p the estimates 

\fz(vk) ~ fz(v )| < ll-ll • ||F(u fc ) - F(v )|| < ll^ll • - u|| -> 0 as k -£ 00 , 

s — r 

which is also a contradiction due to the assumption f z (vk) < f z (v) — e. This justifies the 
lower semicontinuity of f z on the space X for any fixed z £ X. 

Now we claim that T is monotone. To proceed, pick two pair ( Ui,Vi ) G gphT and get 

(Vi, uf) £ gph F with y t := v t + su h i = 1,2. 


Applying the mean value inequality (j2.8j) to the l.s.c. function f z tells us that 

\(ziU 1 -u 2 )\ = \f z (yi)-f z (y 2 )\ < llx/i 2 / 2 1| sup {||w|| | w£df z (y), y £ [yi,y 2 ]+elB} (3.6) 

with [ 2 / 1 , 2 / 2 ] := {A 2/1 + (10 — A)2/ 2 | A £ [0,1]} and fixed £ > 0. Sine edf z (y) = 0 if y dom f z , 
it suffices to consider the case of y £ dom f z D ([ 2 / 1 , 2 / 2 ] + — dom F D ([ 2 / 1 , yf\ + sJB) in 

(13.6p . Take any y from the latter set and observe that 

w £ D*F(y)(z ) whenever w £ df z (y). (3.7) 


Indeed, it follows from the definition of the regular subgradient w £ df~(y ) that 

linnnf ^ ~ / - M ~ f’ V ~ g> > 0, 
v->y ||u — y || 

which can be equivalently written by the construction of f z in (13.51) as 

liminf ~ ( Z > F (V)) - (w,v-y) > 


dom F 
V y 


\v - y II 
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The latter readily implies that 


liminf ( y M ~-/> > 0 . 

Hence we get from (12. ip . (12.3j) , and (12.4[) that 

(w,-z) G N((y,F(y)y,gphF)^we D*F{y)(z) = D*(T + sl)~\y)(z) 

and therefore —z G D*(T + sI)(F(y), y)(—w). It easily follows from the coderivative sum 
rule in pHy Theorem 1.62] that 

— z + sw E D*T ( F(y),y — sF(y )) (—w). (3.8) 

Combining this with (13.3ft tells us that (—z + sw, —w) > 0, which yields 

|p|| • \\w\\ > ( z,w) > s||w|| 2 (3.9) 

and implies furthermore together with the estimate (13.61) that 

\(z,ui ~ u 2 )\ < s -1 |MI • \\yi ~ 2/2 1 |• 

Since this inequality holds for all z G A", we get 

||tti - M 2 II < s_1 ||2/i - S/ 2 11 = s” 1 |Kh + stti -v 2 - su 2 \\ 
and then deduce by the elementary transformation that 

s 2 ||«i - w 2 || < ||(fi - v 2 ) + s(ui - u 2 )|| 2 = IK - ^ll 2 + 2(ni - v 2 ,ui - u 2 ) + s 2 ||«i - ^ 2 1| 2 - 
Therefore we arrive at the inequality 

0 < -'-||u 1 - u 2 || 2 + (vi - v 2 ,u± - u 2 ) for any s > r. 

o 

Letting there s —> 00 shows that 

0 < (iq — v 2 ,Ui - u 2 ) for all (wi,Ui), (u 2 ,v 2 ) G gph T 


and thus verifies the monotonicity of T. 

It remains to prove that T is maximal monotone. Since T is proper, there exists a pair 
(u o,Vq) G gphT such that 

u 0 = (T + s/) _ 1 (?/o) with y 0 := v 0 + su 0 . 

Applying again the mean value inequality (j2.8[) to the function f z from (13.5ft . we have 

I fz(y) - fz(yo)\ < II?/- 2 / 0 II sup{||w|| I w G df z (x), X G [2/, 2/0] + eB) (3.10) 

for any y G X. It follows similarly to (13.9ft that ||iu|| < 1|-^|| for all w G df z (x) with 

x G dom F D ([y,yo\ + sJB). This together with (13.10ft gives us the estimates 

I f z (y) - fz(yo)\ < s _ 1 ll-ll • II y - 2/0II- 
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Hence f z (y ) < oo and so F(y ) ^ 0 for all y G A", which means that dom (T + s/)^ 1 = A". 
Employing now the classical Alinty theorem (see, e.g., [61, Theorem 4.4.7 and Remark 4.4.8]) 
and taking into account the monotonicity of T justified above, we conclude that T is 
maximal monotone and thus complete the proof of the theorem. □ 

Our next goal is to obtain another version of the coderivative characterization in The¬ 
orem 13.21 with replacing the global hypomonotonicity of T in assertion (ii) therein by a 
certain local hypomonotonicity. Besides being interesting for its own sake, it is needed for 
the subsequent applications in Section 4 to characterize convexity and strong convexity of 
lower-C 2 functions. In fact, for these purposes we need to modify the conventional notion 
of local monotonicity and hypomonotonicity, which are dealing with neighborhoods in the 
product space X x A"; see, e.g., [281 13l] with more references and discussions. The notions 
we use in what follows concern neighborhoods only in the domain space A". Such a lo¬ 
cal monotonicity has been considered in [38], Example 12.28]. We will name the domain 
versions as semilocal monotonicity and hypomonotonicity, which reflects their nature and 
distinguishes them from their fully localized product counterparts. 

Definition 3.3 (semilocal monotonicity and hypomonotonicity). We say that the 
mapping T: X X is semilocally hypomonotone ( resp. semilocally monotone) 
at u G dom T if there exist a neighborhood U of u and a number r > 0 (resp. r = 0) with 

(ui - v 2 ,ui - u 2 ) > -r\\ui - u - 2 \\ 2 for all (u 1: vf), (u 2 , v 2 ) G gphT fl (U x A). (3.11) 

Given a set 0 C X, we say that T is semilocally hypomonotone (resp. monotone) ON 0 if 
it is semilocally hypomonotone (resp. monotone ) at every point u G 0 fl domT. 

Establishing the desired semilocal version of Theorem 13.21 requires an additional con¬ 
vexity assumption on the domain of T, which is shown below to be essential by providing a 
counterexample. To proceed in this direction, we first present the following lemma proved 
in [23] Theorem 5] by similar arguments for single-valued operators. 

Lemma 3.4 (semilocal monotonicity of set-valued mappings with convex do¬ 
mains). Let T: X X be a semilocally monotone mapping on X, and let its domain 
domT be convex. Then T is ( globally ) monotone on X. 

Proof. Pick any (u^, iq), (u 2 , v 2 ) G gphT and get [ui,u 2 ] C domT by the convexity of 
domT. Since T is semilocally monotone, for each x G [ui,w 2 ] there is > 0 such that 

{]J\ ~ y 2 ,xi ~ x 2 ) > 0 whenever (x 1 ,y 1 ), (x 2 ,y 2 ) G gphT n (int B lx (x) x A). (3.12) 

The compactness of [ui.u-f allows us to select Xi G [wi,m 2 ], i — 1,... ,n satisfying 

n 

[ui,u 2 \ c U int 1Bj x . ( Xi j). 

i= 1 

Thus we can find 0 = to < t± < ... < t m = 1 such that for each j G {0,..., m — 1} it holds 
[uj, Uj+i] C int dB lx , (x^) with some ij G {l,..., n}, 
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where Uj := U\ + tj(u 2 — Mi). Since Uj G [mi,m 2 ] C domT for each j G { 0 ,... ,m}, there 
exist Vj G T(v,j) with u 0 = v i and M m = u 2 . It follows from (13.12p that 

{tj+l 1 M 2 Mi) (Pj-t-1 Vj , Mj44 Mj) ^ 0, 

which implies that (mj+i — Vj, u 2 — Mi) > 0 whenever j G {0,..., m — 1}. Hence we get 


m— 1 

(m 2 - Ml, M 2 - Ml) = y> J+ i - ‘Vj , M 2 - Ml) > 0 
5=0 

and thus verify the global monotonicity of the operator T. □ 

Now we are ready to obtain a semilocal counterpart of the coderivative characterization 
in Theorem 13. 2l under the convexity assumption on dom T. Example 13. Gl below demonstrates 
that the latter assumption cannot be dropped. Since the proof of the following theorem is 
similar in some places to that of Theorem 13.21 we omit the corresponding details. 

Theorem 3.5 (regular coderivative and semilocal hypomonotonicity characteri¬ 
zation of maximal monotonicity) . Let T : X X be a set-valued mapping with closed 
graph and convex domain. The following assertions are equivalent: 

(i) T is maximal monotone on X. 

(ii) T is semilocally hypomonotone on X and satisfies the regular coderivative condition 
(13.31) for any (u,v) G gphT. 

Proof. Implication (i)^=^(ii) follows from Theorem 13.21 To verify the converse implication, 
suppose that condition (13.3|) holds and that T is semilocally hypomonotone. The latter 
allows us to find, for each u G dom T, positive numbers 6 and r such that 

(mi — v 2 ,ui—u 2 ) > —r||Mi — m 2 || 2 whenever (ui, ui), (u 2 , v 2 ) G gphTC](lBs(u) xX). (3.13) 

Take any s > r and define the mapping F : X X by gphF := gph (T + s/) -1 fl 
(.X x JBs{u)). Picking arbitrarily pairs (Mj,M*) G gph F, i — 1,2 we have (m^m* — sm*) G 
gphT D (JBs(u) x A"). It follows from (j3.13|) that 


(ui - SMi - u 2 + SM 2 , Ml - M 2 ) > —r||Mi - m 2 ||". 

Similarly to (13.61) we deduce from the latter that 

||mi — m 2 || < ||m! — m 2 || for all (vi,ui),(v 2 ,u 2 ) G gph F. (3.14) 

s — r 

This implies that F is single-valued and Lipschitz continuous on domT. For any fixed 
vector z G A" we also define the function f z : X —> JR as in (13.51) and prove similarly to 
Theorem 13.21 that f z is lower semicontinuous on X. 

Now pick arbitrary pairs (ui,Vi), (m 2 ,m 2 ) G gphT fl (int JB^iu) x X) and fix v G T(m). 
Then F^yf) = Uj G JB$(u) with y t := Uj + sm*. Applying the mean value inequality (12.8j) for 
any £ G (0, y/s) tells us that 

|(z,ui-u 2 ) = \fz(yi)-fz(V 2 )\ < II 2/1 — 2 / 2 1| sup {\\w\\ | w G df z (y),ye [y 1 ,y 2 ]+eB}. (3.15) 
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Similar to (13. 7 J) we get df z (y ) C D*F(y)(z) for all y G dorri F fi ( [2/1, 2 / 2 ] + eiB) and then 
for any y G dorri F fi ( [2/1 , 2 / 2 ] + £-2?) find some y 0 G elB and t G [0,1] satisfying y = 
ty\ + (1 — t)y 2 + yo- Since F(v + su) = u, it follows from (13.141) that 


II F(y) - nil = WF^yx + (1 - t)y 2 + y 0 ) - F(v + su) 
<- \\tyi + (1 - t)y 2 + Vo-v- s«|| 


< 


< 


< 


s — r 

1 

s — r 
1 

s — r 

1 

s — r L 
1 

s — r 

1 

s — r 


||t(wi + sui) + (1 - t)(v 2 + su 2 ) + yo - v - sh|| 

II^Oi - v) + st(u 1 - u)) + (1 - t){v 2 - v) + s(l - t){u 2 - u) + t/oll 
t||fi - v\\ + (1 - t)\\v 2 - v\\ + st\\ui - u\\ + s(l - t)\\u 2 - h|| + \\y 0 


max 


max 


{||fi — n||, ||n 2 — n||} + e + 
{||fi - h||, ||u 2 - u||} + Vs 


s — r 
s 

H- 

s — r 


(3.16) 


■ max {||ui — n||, ||n 2 — tt||} 

■ max {||ni — h||, ||n 2 — h||}. 


Since the choice of (ni,Ui), (u 2 ,v 2 ), (u,v) G gphTfl (int JBs(u) xX) was independent of the 
parameter s > r and by max{||ui — h||, ||n 2 — h||} < S, we can find M so large that 

1 s 

- -- max {||t>i — h||, ||n 2 — h|| + \/s} H-- max {||ni — x||, ||n 2 — x||} < S if s>M. 


This together with (13. 16ft ensures that F(y) G int JBs(x) and thus 

N((y, F(y )); gphF) = N((y, F(y))-, gph (T+siy^X xlB 5 (x)) = N((y , F(y)); gph (T+sI )' 1 ), 
which clearly implies in turn the equality 

D*F(y)(z) = D*(T + sI)-\y,F{y))(z). 

Similarly to (13.8ft . for any w G df z (y ) C D*F(y)(z) we get from the latter that — z + svj G 
D*T(F(y),y — sF(y))(—w). ft follows from (13.3[) that (—z + sw, —w) > 0, which yields 

||z|| • ||tu|| — { z i w ) — s||n;|| 2 , i.e., H^ll > s\\w\\. 

This together with (j3.15[) tells us that 

{z,Ui — u 2 ) < s _ 1 ||r/i — 2/2 1 | • |M|. 


Since the latter holds for any z G A", we have 

||«i - n 2 1| 2 < s _2 ||r/i - y 2 1| = s“ 2 ||ui + sui - v 2 - su 2 1| 2 = s _2 ||(ui - v 2 ) + s(ui - u 2 )|| 2 

and hence arrive at the estimate 

0 < -||ni — n 2 || 2 + 2(t>i — v 2 , u\ — u 2 ) when s > M. 
s 

Letting there s —* 00 shows that 

0 < (nr — v 2 ,ui - u 2 ) for all (m, V\), (u 2 , v 2 ) G gphT fl (int B 5 (u) x A"), 
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which verifies the semilocal monotonicity of T at any u G dom T. Since dom T is convex, 
Lemma 13.41 tells us that T is globally monotone. Now we are in a position to apply 
Theorem 13.21 and conclude therefore that T is maximal monotone on X. □ 

It is well known in monotone operator theory that the maximal monotonicity of T always 
yields the convexity of the closure of the domain cl(domT); see, e.g., [2} Corollary 21.12], 
This naturally gives a raise to the question: whether Theorem l3.5l is true when the condition 
on the convexity of domT is replaced by the convexity of cl (domT)? The following simple 
example shows that it is not true and consequently that the convexity assumption on dom T 
in Theorem 13.51 cannot be dropped. 

Example 3.6 (semilocal monotonicity does not yield the convexity of the do¬ 
main) . Define the mapping T : IR IR by 



Observe that gphT is closed, T is semilocally monotone on IR , domT = iR\{0} is non- 
convex while cl(domT) = 1R is convex. Moreover, it is obvious that assertion (ii) of 
Theorem 13.51 is valid, but T is not globally monotone on 1R. 

The next theorem provides other coderivative characterizations of maximal monotonic¬ 
ity, where the regular coderivative condition (I3.3j) is replaced by the positive-semidefiniteness 
conditions imposed on the mixed limiting coderivative (12.5jl . These characterizations are 
clearly equivalent to those presented in Theorems 13.21 and 13.51 but in this paper is more 
convenient for us to derive them by passing to the limit in (13.3p . Note that the limiting 
coderivative characterizations have a strong advantage in comparison with (13.31) due to 
well-developed calculus rules for (12.5j) : see Remark [3.81 and Section 5 for more discussions. 

Theorem 3.7 (limiting coderivative characterizations of maximal monotonicity). 

Let T \ X X be a set-valued mapping with closed graph. The following are equivalent: 

(i) T is maximal monotone on X. 

(ii) T is hypomonotone on X and for any (u,v) G gphT we have 

(z,w) > 0 whenever z G D* M T(u,v)(w), w G X. (3.17) 

If in addition the operator domain domT is convex, then the ( global ) hypomonotonicity in 
assertion (ii) can be equivalently replaced by the semilocal one. 

Proof. Implication (ii)=^(i) is straightforward from Theorem 13.21 due to 

D*T(u,v)(w) C D* m T(u,v)(w) for all (u,v) G gphT, w G X. 

Thus (13.3p follows from (I3.17p , and T is maximal monotone by Theorem 13.21 

To justify the reverse implication (i)=^(ii), suppose that (i) holds, and so (13. 3 p is valid 
due to Theorem 13.21 Picking any ( u , v ) G gphT and z G D* M T{u , v){w) and using definition 

(12. 5p of the mixed limiting coderivative, we find sequences ( Uk,Vk ) SF> V ( u,v ) with Zk A 0 
and Wk — y u) satisfying Zk G D*T(uk,Vk)(iVk ) for all k G IN. It follows from (13.3p that 
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(. Zk,Wk ) > 0. Letting k —* oo and taking into account that sequence {wk} converges 
strongly in X give us that (z,w) > 0, which verifies (I3.17p . 

If dom T is convex, we proceed in the same way with replacing hypomonotonicity by 
semilocal hypomonotonicity and using Theorem 13.51 instead of Theorem 13.21 □ 

Remark 3.8 (advantages of limiting coderivative characterizations). Although the 
coderivative conditions (j3.3[) and (13.171) married with the corresponding hypomonotonicity 
give us the equivalent characterizations of maximal monotonicity, the limiting coderivative 
condition (13. 17j) has clear advantages in comparison with the regular coderivative one (13.31) . 
This is due to the well-developed full calculus for the limiting coderivative (in contrast to 
its regular counterpart) presented in the first volume of |25j . The comprehensive calculus 
rules developed for (12.51) allow us to deal with various compositions of set-valued and single¬ 
valued mappings and to establish maximal monotonicity of structurally composed operators 
under the validity of the corresponding qualification conditions. We refer the reader to both 
volumes of [25] for numerous applications of the coderivative calculus to different issues of 
variational analysis, optimization, and control while not related to monotonicity. 

Note also that a similar full calculus is available in [25] for the normal limiting coderiva¬ 
tive, which is defined by scheme (12. 5 p with replacing the strong convergence y —> w therein 
by the weak convergence in X. However, the corresponding positive-semidefiniteness con¬ 
dition in terms of the normal coderivative is only sufficient (together with the imposed 
hypomonotonicity) for the maximal monotonicity of T outside finite-dimensional spaces. 
The proof of the necessity part given in Theorem 13.71 does not hold true for the normal 
coderivative, since we cannot pass to the limit in the inequality ( Zk,Wk ) > 0 when both 
sequences {zk} and {wk} converge only weakly in A" as k —> oo. 

The following one-dimensional example shows that the hypomonotonicity conditions in 
(ii) in Theorem 13.21 Theorem 13.51 and Theorem 13.71 are essential for the obtained coderiva¬ 
tive characterizations of maximal monotonicity. 

Example 3.9 (hypomonotonicity is essential). Given n > 0, define the set-valued 
mapping T: JR ^ JR with full domain given by: 

T(x ) := kx + [0,1] for all x G JR. 

It is easy to calculate directly by the definitions (or using elementary calculus) that 

if w = 0, v — nu G (0,1), 

if w > 0 , v — ku = 0, 

if w < 0, v — ku — 1, 

otherwise. 

Thus both coderivative conditions (13.3p and (I3.17P are satisfied. However, T is not mono¬ 
tone. The reason is that this mapping is not semilocally hypomonotone. 

As consequences of the obtained results, we derive in the next corollary the correspond¬ 
ing regular and limiting coderivative characterizations of strong maximum monotonicity 
for set-valued mappings in Hilbert spaces. Recall that T : X X is (globally) strongly 


D*T(u,v)(w ) = D*T(u,v)(w ) = < 


({ 0 } 

{kw} 

{hiw} 
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maximal monotone on X with modulus k > 0 if it is maximal monotone and the shifted 
mapping T — kI is monotone on X. It follows from the classical Minty theorem that T is 
strongly maximal monotone with modulus k if and only if T — kI is maximal monotone. 

Corollary 3.10 (coderivative characterizations of strong maximal monotonicity). 

Let T : X X be a set-valued mapping with closed graph. The following are equivalent: 

(i) T is strongly maximal monotone on X with modulus k > 0. 

(ii) T is hypomonotone on X and for any (u,v) E gphT we have 

(z,w) > k||w|| 2 whenever z E D*T(u,v)(w), w E X. 

(iii) T is hypomonotone on X and for any (u, v ) E gph T we have 

(z,w) > n\\w\\ 2 whenever z E D* M T(u,v){w), w E X. 

If in addition the operator domain domT is convex, then the (global ) hypomonotonicity in 
assertions (ii) and (iii) can be equivalently replaced by the semilocal one. 

Proof. Define S := T — kI and get from the corresponding coderivative sum rules in [25l 
Theorem 1.62] the equalities 

D*T(u,v){w) = D*S(u,v — ku)(w) + kw and D* M T(u,v){w ) = D* M S(u,v — ku)[w) + nw 

for all (u,v) E gphT and w E X. Thus the validity of (ii) (resp. (iii)) for T is equivalent 
to the fulfillment of all the conditions in Theorem I3.2f ii) (resp. in Theorem I3.7f ii)) for 
the operator S'; it is obvious for hypomonotonicity. Applying now Theorem 13.21 and Theo¬ 
rem 13.71 respectively, we get that either assertion (ii) or (iii) of this corollary is equivalent 
to the maximal monotonicity of S. Since the latter is equivalent to the strong maximal 
monotonicity of T with modulus k, we complete the proof of the corollary. □ 

Note that a certain localized regular coderivative characterization of the local strong 
maximal monotonicity for set-valued mappings with respect to product neighborhoods (see 
the discussion right before Definition 13.3j) has been recently obtained in [2S1 Theorem 3.4] 
while being independent of the global characterizations in Corollary 13.101 

4 Second-Order Characterizations of Convexity 

In this section we apply the obtained coderivative characterizations of maximal mono¬ 
tonicity and second-order subdifferential constructions to characterize convexity and strong 
convexity for the remarkable class of lower-C 2 functions. 

Recall [3H, Definition 10.29] that a function /: lR n —tlR is lower-C k with k E IN U{oo} 
if for each x E IR n there is a neighborhood V of x on which ip admits the representation 

f(x) = max f t (x), xEV, 

where the functions f t are of class C k on V, where T is compact, and where ft(x) and all 
their partial derivatives in x through order k depend continuously on (t, x) E T x V. This 
class of functions introduced by Rockafellar [37] is among the favorable classes of functions 
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in variational analysis and optimization. Many nice properties and equivalent descriptions 
of such functions can be found, e.g., in mmm- As shown in [38] Corollary 10.34], for 
each k > 2 the class of lower-C^' functions coincides with the class of lower-C 2 functions. 
However, the latter is a proper subclass of lower-C 1 functions. In fact, between the class 
of lower-C 1 functions and the class of lower-C 2 functions there are the classes of lower-C 1 ’" 
functions, 0 < a < 1, which have been studied recently by Daniilidis and Malick [14] . 

The next theorem is the main result of this section, which provides complete second- 
order characterizations of convexity for lower-C 2 functions in finite dimensions. 

Theorem 4.1 (second-order subdifferential characterizations of convexity for the 
class of lower-C 2 functions). Let f: lR n —> 1R be a lower-C 2 function. Then the following 
assertions are equivalent: 

(i) / is convex on M n . 

(ii) For each (u, v ) G gph df we have the condition 

(z,w) > 0 whenever z G d 2 f(u,v)(w), w G FR n . (4-18) 

(iii) For each (u,v) G gph df we have the condition 

(z,w) > 0 whenever z G d 2 f(u,v)(w), w G M n . (4.19) 

Proof. To verify implication (i)=^(ii), observe that the subdifferential operator dip is 
maximal monotone for any convex l.s.c. function by the classical result of convex analysis. 
Hence condition (I4.18P follows from implication (i)=^(ii) in Theorem 13.71 by construc¬ 
tion (12.7p of the second-order subdifferential in finite dimensions. The next implication 
(ii)=^(iii) of the theorem is obvious due to the inclusion 

d 2 f(u,v)(w) C d 2 f(u,v)(w) for every ( u,v ) G gph <9/, w G IFF. 

To prove finally implication (iii)=^(i), suppose that / is lower-C 2 and that condition 
(I4.19P holds. It follows from [38] Example 12.28] that the subdifferential mapping T(x ) : = 
dip{x) for the lower-C 2 function ip is semilocally hypomonotone with dom T = M n . Then 
(c) amounts to saying that for any (u,v) G gphT we have 

(z,w) > 0 for all z G D*T(u,v)(w ), w G lR n . 

Since the set dip is closed in this setting, we deduce from Theorem I3.5l that T is monotone, 
and thus ip is convex by the result of [13]; see also [25, Theorem 3.56]. □ 

Remark 4.2 (discussion on second-order subdifferential characterizations of con¬ 
vexity). The following comments are in order: 

(i) Consider the pointwise maximum of finitely many C 2 functions 

/(x) :=max{/i(a:),...,/ m (i)}, x G 1R, (4.20) 

which surely belongs to the class of lower-C 2 functions. Based in the recent precise calcu¬ 
lation [Hj of the second-order subdifferential d 2 ip of (14. 20p via the generated functions ipi 
and the appropriate index subsets of {1,..., m}, we can express the second-order charac¬ 
terization (14.18P entirely in terms of the initial data of (14.201) . 
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(ii) Second-order subdifferential characterizations of convexity have been recently ob¬ 
tained in [3 El [9, TOJ [IT] for functions that may not be necessarily lower-C 2 , and thus they 
are generally independent of Theorem 14. 11 Note also that it is not clear by now whether the 
characterizations of Theorem 14.11 hold if / is merely lower-C 1 or lower-C 1 ’" with 0 < a < 1. 

Finally in this section, we present the second-order sub differential characterizations of 
strong convexity for lower-C 2 functions, which can be treated as consequences of Theo¬ 
rem 14.11 Recall that / is strongly convex on lR n with modulus k > 0 if 

f(t\x + (1 - A )y) < tf(x) + (1 - A )f(y) - ^A(l - A)||x - y\\ 2 

for all x,y G JR and A G (0,1). It is well known that the strong convexity of / is equivalent 
to the convexity of the shifted function 

g(x) := f(x) - ^\\x\\ 2 , x G M n . (4.21) 

Corollary 4.3 (second-order subdifferential characterizations of strong convexity 
for lower-C 2 functions). Let f: lR n —>• JR be lower-C 2 . The following are equivalent. 

(i) / is strongly convex on M n with modulus n > 0. 

(ii) We have the second-order subdifferential condition 

( z,w) > n\\w\\ 2 for all z G d 2 f{u,v)(w), (u,v) G gph df, w G lR n . 

(iii) We have the modified second-order subdifferential condition 

{z,w) > k\\w \\ 2 for all z G d 2 f(u,v)(w), (u,v) G gph df, w G lR n . 

Proof. It can be derived from Theorem 14.11 by applying it to the shifted function (14.211) 
and taking into account the obvious subdifferential relationship 

dg(x) = df(x ) — kx, x G lR n . 

On the other hand, we can justify the results by applying the characterizations of strong 
maximal monotonicity from Corollary 13.101 similarly to the proof of Theorem 14.11 □ 


5 Concluding Remarks 

The main results of this paper provide coderivative characterizations of maximal mono¬ 
tonicity for set-valued mappings under one of the following assumptions: (a) the mapping 
is globally hypomonotone without imposing the convexity of its domain, and (b) the map¬ 
ping is semilocally hypomonotone with convex domain. Example 13.91 shows that removing 
hypomonotonicity may destroy these characterizations. However, it is proved in [9] that 
for single-valued mappings hypomonotonicity can be replaced by continuity. Thus the first 
open question is to clarify what is common for both hypomonotonicity of set-valued map¬ 
pings and continuity of single-valued ones. We intend to develop general results in this 
direction, which unify these two requirements. 
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The second area of the promising future research is to find an umbrella, which covers 
all the second-order sub differential characterizations of convexity for functions developed 
in the previous investigations [7, IS, Q [10, 11] and those obtained in Theorem 14. II for lower- 
C 2 functions. So far we can deduce [7] Theorem 4.1] from Theorem 14.11 while the other 
major results of the aforementioned papers seem to be independent. It is also desired to 
obtain second-order subdifferential characterizations of convexity for the important classes 
of lower-C 1 and lower-C 1,Q (0 < a < 1) functions. 

The third and probably most important direction of the future research is employing 
the limiting coderivative calculus to derive from the pointwise coderivative characteriza¬ 
tion (13. 17ji verifiable conditions for preserving maximal monotonicity under various com¬ 
binations (including sums, compositions, etc.) of set-valued and single-valued maximal 
monotone operators. The classical result in this vein is Rockafcllar’s theorem [35] about 
maximal monotonicity of sums of maximal monotone operators under certain interiority or 
local boundedness assumptions. It seems that the coderivative characterizations of max¬ 
imal monotonicity obtained in our paper open a new gate (in Hilbert spaces so far) to 
proceed in this direction via verifying qualification conditions that ensure the validity of 
the corresponding coderivative calculus rules from [25] . 
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